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ABSTBACT 

The Phase 2 effort of a task to develop a new student 
critique fora for the Air Training Coaaand CATC) is descrilied. Phase 

1 recoaaended the further developaent of group specific noras for 
officers, MCOs, and airaen enrolled in technical training schools. In 
Phase 2 additional iteas were added to the prototype fora resulting 
in a later version of the critique fora that had a reliability of 
•80. The Phase 2 version of the Student Critique Fora (SCF) was 
adainistered to a saaple of over 1,800 students enrolled in technical 
training courses at six Air Force bases. Moras were gathered 
separately for officers, MCOs, and airaen. Seven s^les eaerged froa 
a factor analysis of the Phase 2 version of the SCF: Instructor 
coapetence. Study Snvironaent and Testing, Specialty Training, 
Training lapressions, Classrooa Facilities and Environaent, Training 
Devices and Audio Yisual Aids, and Training Baterials Adequacy. Phase 

2 of this effort «as subsequently initiated to develop the coaputer 
software to score and interpret the SCF. (Aothor/BC) 
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This report describes the Vime II effort of a tads to develops new student critique fom for the Air Trainii^ 
Command (ATC). Pha^ I oi this effort, as reported hi AFHRL-TR-70-37, De^Iofm^t of hydicnnetrk Measures 
of Student Attitudes Toward Technical Trainb^: Reliabflity and F^ori^ Validity, reccmumncted &e furt^ 
deveiopn^nt of group ^ifk norms fc. ofAcei^ NCOs, and airmen enioBed in Air Fcn^ tedskal trabibig schoute. 
In Phase TI addittonal it^s were sdii^ to the prototyj^ form re^lth^ in a later versiM of ^ critique form that 
had a reliability of The Riase II ver^ of the Stu<tent Critiqi^ Form (SCF) was iKlminMeied to a s^mtqrie of 
over 1800 students enrolled in tedinical train^ ccmrses at ^ Air Force t^ses* tkatm were gathered s^mrately fc^ 
oHkers* NCOs, and airmen ^nce additional analyses indteat^, a« did the Phase I andyses, Aat tt^ three groups were 
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signtffcwitly different with respect to attitudes as measured by the SCF Seven scales emerwd fmm of-.. i. ^ 
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the computer software to score and interpre?the SCF." subsequently initiated to develop 
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SUMMARY 



Piebtem 

The purpose of th^ study was to comptete Phase II ai^ develop a new student critlqi^ fonn for die 
Air Trainif^ Comnund (ATC). The si^ifk c^jectives of Phase II were: (a) to gyrate more items for the 
prototype sc^ deveio{^ in fliase l\ in order to increase scale relfaib3ity;(fr) to administer the Biase II 
Student Crit^ue Form (SCF) to sufHcfently large samples of officers, NCOs, and ainmn enrcdted in 
technical training co»;<^s; (c) to factor analy;^ the SCF again to in^re scale unidinwnsicma]ity;ai^(<0 to 
norm the SCF separately for officers, NCOs, and airmen. 

The SCF was ^ministered to a sampte of 1 «82S offkrers, NCOs, and airmen Vskis^ technical training 
cci'rses at six Air Forc^ ^ases. Scores were repOTted for eadi rank and Air Force Imse wheie the SCF was 
admmistered as indivtduJ average scores and scate scc»^ Norms were coHec^ for totd scale score* 
suhscale scores, and item scores f&c each of die three rank ^oups and each base, and re(K>rted in deciles. 
Factor an^yses were run to dieck the unkSimensionality of the SCF whSe other analyses were run to 
determine if there were s^if icant differences betw^n the three Air Force rank grcHips. 

Results 

Factor analyses established that there were seven unidin^n^onal scales in tl^ SCF: Instructor 
Competence, Study &ivironn^nt and Testing, l^>eci^ty Training, Training Imj^es^ons, Classroom 
F^Oities and Environment, Trainit^ Devi^ and Audk>-V^ial Ai(b, and Trainti^ Materials Adequ^y. An 
analysis of SCF scenes indkrated diat there w^ sapiifkant different between the SCF scores of offkers, 
NCOs, and airmen on e^ scale of the SCF wlien compared by nmk and base «1iere the SCF was 
administered. 

CondnsioBS 

A 69 item critk)ue form was deve}<^>ed that con^ted of seven unidfanen^md scales. Norms were 
collected for the critk)ue form for each base where the form was admintetered and for each of the three 
rank groups at each base. The nwms will enable users of the SCF to interpret the attitiuks of students 
enrolled in technical training ccHirses relative to an appropriate peer group. Hie next phase in the 
development of a new critk^ue form calls for the developn^nt of the apprtpriate computer software to 
score and interpret the SCF. Upon completion of Phase HI of this prcgect, the SCf wOl be rec(»nmended 
for ATC use. 
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DEVHLOPMENT OF PSYCmMbrRIC MHASURES 
OF STUDENT ATTITUDES TOWARD TECHNICAL TRAINING: 
NORM CROUP REPORT 



I. INTRODUCTKIN 

Many educational itistitutions, as well as the 
United States Air Force, have turned lo student 
critique ratings as an aki in evaluatit^ the relative 
effectivene^ of insinictors and instructionai 
programs. The adequi&y of students as judges of 
the competency of instmctors and tnstnictional 
programs remains a controversial issi^. Two m^or 
conHfeting viewpoints on this i^ie twre cited by 
Coffman (1^54). He !»ggested that (a) students 
are Ufo incxper^nced and subjective to make 
curate, bias free rating and {b) student ratings, 
fegard!e<is of bias, provide valuabte infomiatbn 
concerning instructor and amt^ effectivene^ 
The opinions and attitudes expressed by students 
on critique forms can obvkHi^y be due to many 
different factors. Students may have different 
frames of reference, different expectatk>ns for a 
course, and different ideas as to the relevaiK>' of a 
partkuiar course. Inasnuch as I^ of motivation 
or insincerity may lead a student to improperiy 
respond to a critkiue form, the interpretation of 
su-U data must of necessity be tempered with 
knowledge of the limitations of stucfent ratings. 
The fact still remains that students do observe 
instrucfion more than anyone el^ and are, there- 
fore, the most logical candidates for feedbatk on 
training and the training environment. Even 
fhcni^ serious problems have been asswkited with 
students* ratinp of the learning environment, the 
preponderance of studies, as reported by Coffman 
(1954), indicated titat student ratings are sottc- 
what reliable, need not nccessarUy be influenced 
by the halo effect (the tendency to rate indhfkiuds 
too hig^ or too low on the basis of one out- 
jianding trait) and can quite similar to the 
ratings of seasoned alumni. 

Anikeef (I9S3) conducted a study to 
determine the utifiiy of student evaluatk)n of 
faculty members and alsc^ to determine the effects 
of instnictor's grading leniency and the ratir^s the 
instmctor received. A not too surprising finding of 
the study was that the ratir^ received by the 
instructor was pi^sitiveiy correlated with die grade 
received by the student. McKeach^ and Sdcnnan 
(1958) were also interested in studying the 
effectiveness of students* rating of instructors. In 
attempting to ascertain the validity of students* 
ratings, they rcast^t^d that one criterkm of 
instmctor effectiveness wcnitd be his abdity to 
stiiHuiatc interest tn the subject, as measured by 



comparing students' rating of instructors igatnst 
the p^'rcentage of stucfents «4)o etected to take 
advatwed courses in the san^ ^fefwt area. The 
re^ts indicated that ratinp by students of 
instmctcH^ were signifkantty correlated with the 
f^rcentage of con inuing sttii!ents in two of the 
five sen^sters. Althou^ the ratii^ were not 
correlated with the criterion of ccxtrse selection in 
the san« subject area in a m^ority of cases, the 
ratii^ dki provMe vduabte faiforn^tton fc^ 
instructors who widted to improve their per- 
fom^nce. tn a stmSar *^in, Co^ove (IV59), 
develc^d a student rattag technk|M c^abte of 
producatg a diagnmtk instructor profBe that 
c<HiId be used by infractors as an aid in kkn* 
tify tng and evaluating strmg and weak elements of 
teadiing performance. TTie use of the proTfc 
iqipro^h appeared to be veiy helpful by providing 
a diagnostic starting po^t fw instructed 
improvenmit. 

Weaver (I960) dossed s<mie of the problems 
that may associated with student critiques, 
including the overall reluctance of bistructors to 
being rated, the relaticmiiitp of the rating given an 
instnictof or cwt^ and the graite received for that 
course, and the influence of the stuitent^s per- 
ception of his own m^hic^ment upon the rating he 
assies an instructor or cout^. An invest^tion 
was subsequently macte into the relaticmi^ip of the 
rating received by the instmctc^ and the grade the 
student expected to receive in the course. The 
question of \%1iether student cr tk:ism k directed 
towards both instructor persotiaiity and teaching 
skill or if the halo effect attenuates students* 
criticism of both of these variables was investi* 
gated. Ratings of personality were ncrt fmitKi to be 
related to expected grade, su^t^g that the 
popularity of an inf ructor doi*s not necessarily 
influence his rat^gs to any s^ifkant extent. 
Odier research (Hdlander, l%S) in the area (rf 
leaderd)ip agrees, Hius, even thcHi^ an instructed 
may attain a d^rce pq)ularity based on 
peruH)ality, stucfaznts may judge his effectiveness 
based upon another set of criteria, e.g., the 
competency of the instructor. 

l^biNls of ^tesartag ^itdeot Attit^ies 

There are various* n%th(»ds avayabte Uiat can 
used to measure student attiiudes and opinions. 
Useful informatkm can often Ik; derii^ by simply 
determining the number of students who agree or 
disagree witli certain stalemcnf<iah<»it the training 
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environment, or by requesting studenu to submit 
short written essiQfs about t^MfV trailing 
situations, Individudized methods sudi as these do 
not. however, provitk suflkkmt infiHtnaticm to 
compare the results of one course with another. 
Summan/atkin ami attempts at standardisation 
when using these methods can be tedious at best. 

The mea^iement of student attinittes provides 
the most useful data when inter- aiK^ mtra^ourse 
altitudes toward a partkmlar course can be 
ccmpar^. Spencer and Aleamoni (197|) state 
that adequate interpretation occurs when {a} 
approprbte attention is given to s^pitaig, relia- 
bOity, and vdidity and (b) the rnne instmment is 
used in a standardized mani^r to n^awre many 
instnictors and instructional prc^ams. Such an 
approach enables investigators to develop norms 
such that instPictors, training manors, and 
supeivisors can make useful ccmparisons, cot^rse 
by course, and snstmctor by instructor. 

Rdevrat Grttoia 

Various limitations involved in the selection 
and construction of auitucte and opinion instni- 
mcnts are also dhcussed by Spencer and AleamcMii 
(1971). Relevant criteria for selectii^ or con- 
^ructmg an appropriate instrument are lated 
below: 

I. AJminisirsiiorL The questionnaire ^ould be 
administered by the instructor him^lf during the 
regular class or exam^ation thne, so that proctors 
and administrators would not be necessary. 

Z Time. It shixitd be ^ort eniHi^ to be 
^ceptable to faculty in regular classes, but \m$ 
enou^ lo insure rdmbility and an ^lequate 
me^re of a wi<te sampte of attitudes. 

J. OmtenL It should measure iho^ q)tnkins 
and attitudes which are devclc^>ed or exht ab<Hit 
the total tnstmctional prc^ram rather than a sin^e 
element therein. 

4. Sa^g. it should be objective, and pref- 
crably machine scorable so fhat the results ccHild 
be returned pmmptly anj scoring could he 
standardized and reliable. 

X Relkibtiiiy. If one wants lo insure that 
scores on the instnimcnl arc true leprcscntatiims 
of the students* c^pinions, such scores must be 
consistent as evidenced by simflar scores repro- 
duced through sul^quent testing of the same 
slutknis racing ihe same insimcttu ami c<Hjrsc. 



6. /nterfmiatkPfL It shuiid yield scores which 
differentbte amcmg mstructic^ pn^ams and 
which can be interpietcd by instructors in such a 
manner that their histniclkmid effectivene^ cari 
be m^noved. It duHild 9s^t in the diagnosis of 
the strengths ami weaknesses of the instructiimal 
prc^ram. 

7. Ueaiim. The attitude m^tsures must 
inciuite thf»e critk^d eten^nts w^kh cimprtse the 
attitudes the stuctent has aid widi^ to express: 
but the meamtring instrument must be capable of 
eliciting ^r^** feel^gs, and not careless or merely 
uKiaXiy ^:^tabie or expected responses. 

Ak Force ^inteit Critique Pkopm 

The Air Fcrce, as welt as other traming 
Yganizations ami educatkmal ^stituttcHts, has 
icMig recc^tz^ the importance of the attitudes 
and c^kiiiNis of its trainees concerning the differ- 
ent a^>ects of the technical training environment. 
These factors may determhie to a great extent how 
a student performs in a course. ViduaUe infor- 
matics cm be gained throu^ the analysis of 
attitudes and (^faiic^s of students enrolled in 
t^hnic^ training ccxtrses. Tf^ Air Force currently 
pthers informatiim of this nature thrcnigh the 
student critique pn^ram. 

The ofa^th^ of this prc^am, as stated in 
ATCR 52-29 (Recent changes to ATCR 52-29 may 
not 1^ reflected.) are: . . .to obtam from students 
constructive critic^ of trainmg, tf^ training 
environntent, and base support facflittes ami 
services. AlthcHigh the students' (^faiions may be 
based ufHin Ihnited background and qu^irK:ations, 
his attitudes and reactions affect his learning and 
must be ccmsklered in evaluating training. A weU 
designed and adminstered critique prc^am 
provkks command rs and ^pervisors useful and 
necessary information for improvement of training 
and the stu^lent enviriinmcnt. (para. 2, 17 July 
1970). 

Feikrko ( 1970) has described the abiwe stated 
objective as a c^sed-toop cybernelk system* 
whea'in feedb^'k from constructive critician 
performs three vital functions: 

1 . In formation - furnishes commanders and 
^pervk^ors w!!h data alnnit the current state of 
the training system. 

2. Reinforcement - it strengthens or weakens 
varicHis notions about the nature of tht' training 
program; and 



3. Motivation - it incites corrective action 
within the ^steni when it is warranted. 

Thus, critfcism provides some of the infor* 
matton that is neceswy for the training system to 
maintain a satisfactory, balarwed level of oper- 
ation. 

ATC Form 736 is currently beir^ i»sed by the 
Air Force to obtain constructive critkrism from 
students. Trainees are encouraged, tnit not 
required, to re^tnl to this form near the end of 
their technical courses or after blocks or units of 
instruction. StUiknt critk)ue data are subsequently 
used by commanders and supervisees to modify 
and improve training and the trainif% envircm* 
ment. Recently, however, the retiaoility and 
validity of die ATC Form 736 have been 
sonwwhat $u^>ect due to several serious deii* 
cienci^s inherent in its format. Sime areas of 
concern are the interpretation of the rating 
categories, the criteria for judgments, and the 
l&el^ood of being mi^d by gross ^^^rcentages 
(for a detafled discussion of these deHctencies, ^ 
Federko, 1970). 

in^mved Stt^leiit Oitiqi» Form 

C(%nizant of the shortcomings of the ATC 
Form 736, the Ail Training Command requested 
that the Air Force Human Resources Laboratory, 
Technkat Training Division, develop an improved 
student critique form. As a result of this request- 
AFHRUTT initiated a three phase program to 
meet the above stated objective. The objective of 
pha^ I was to originate a psychometrkally sound 
me^re of student attitudes towards Air Force 
technical training that could i» objectively 
analyzed and interpreted. A thorough review of 
the ph^ I effort is presented by Federko ( 1970, 
1 97 la, 197 lb). In fAas-i II, norms were to be 
developed for the new student cntique form, sudi 
that training mana^*rs and instructors would have 
the necessary data available to interpret the 
attitudes of officers, NCOs, and airmen relative to 
their peers. Upon con^pletion of pha^ II of this 
study, the new form would then be propo^ as a 
replacement for ATC Form 736, Riabe ill calls for 
the devetopn^nt of the appropriate computer 
software t J manage the entire critique program 
within Air Force technical tratnii^ by admin- 
istering, scoring, and interpicting the ^tudeht 
critique forms. 

The purpose i)r this repon is to discuss the 
development ot' the new Muderf cfitkjuc form and 
rcp<irt the as-VKiaicd normative data. 



IL DEVELOPMENT OF THE SC ALt 

In^y SefectioB of H^m 

The initi^d poxA of items used in the devel- 
opment of the Student Critk]ue Form (SCF) was 
selected from student-generated criteria from a 
study conducted by the S(^^ Evaluatkm 
Branch, 3415 Technical $cho<4, Lowry Air Force 
Base (1967). Two critique form prototypes were 
constructed by Federko (1970) from the above 
n^ntioificd student-^nerated criteria. The SCF 
prototypes were constructed uttliztoig a Guttman* 
and a Likert type conf^ui^tion. An cxtenshre 
statisttca! analyst was then perfom^d on ttie SCF 
prototypes (for a detailed discu^n see Federico, 
1970). On the bas» of the aforementkmed 
analysis, the Likert-type conf^uration was 
recomn^n^ for future devclopn^nt* 

Vk&tt Fonnat 

The final SCF developed by Federtco (1970) 
consisted of SS Likert*type statements constructed 
using the methods de^rit^ by likert (1 932) and 
Edwards (1957). The fdtowing datement b an 
example of the Ukert-type format used m the 
SCF: 

Mc^t of your study guides are easy to uinler- 
stand. 

Stroi^iy Strongly 
A^ree Ai;rec Undci-kkd Umagrcc DisagErec 

( ) i ) ( ) (I ( ) 

Student repon^ to an item, such as the one 
above, called for checking one of the five regnm^ 
categories* 

Scale Development 

A principal components f^tor analyst 
followed by a Varimax rotation (Harman, 1%7) 
was used on the initial and later ver^s of the 
SCF. Factor analyses of the initial fonn and a later 
version, vAikh consisted of twenty additional 
items, yielded basicall>' the same seven fetors, 
with the exception of Scale g (Textbook Utility), 
whkh was eliminated. The remaining items bi 
Scale 8 were placed in other scates based upcMi 
their factor loadings and wibjecthre judgments <rf 
their relevance in a partk:ular scale. The items 
repre^nting the^ $^wn factors were grouped 
together and incorporated into the final version of 
the SCF. Each factor consisted of between six ar^ 
sixteen statenMints that were fwnd to ^fine a 
common content area of attitudes towards 



lechnical training and as such ctiuld he considered 
to ci»nstitufe separate unidimensional scales. These 
scales arc listed in Table I . Their names have been 
modified slightly fn>ni the original names to aid in 
infcrpntation of content. The scale intercorre- 
lations listed in Table 2 are based on the SCf 
responses of 1.669 students. The data in Table 2 
indicate that, with the exception of the correlation 
between subsct>res 3 and 6 with subscore 2, all of 
the correlations are significant at the .01 level of 
confidence. This result must be tempered with the 



knt)wledgc that a correlation of only .25 is needed 
for significance at the .01 level, due to the large 
number of subjects. The SCF can then be scored 
to obtain (a) a total score for the entire form and 
(h) a scale score for each of the «ven factors. The 
total score pre^nts an overall estimation of the 
students' altitudes towards technkral training while 
the seven individual scales afford a more detailed 
description and a speclfk: identification of 
potental problem areas. 



Tabie I 




f^imbMf of itenti 
in tue ScAie 



1. Instructor Competent^ 

2. Study Environment and Testing 

3. Specialty Training 

4. Training Impressions 

5. Classroom Facilitws and Environment 

6. Training DevKCsand Audio-Visual Aids 

7. Training Materials Adequs;y 

Total 



13 
10 
8 
7 
6 
16 
9 

69 



Tabie2. Inter-Condations Among Scales 
of the SCF 



II 



ill 



IV 



IV 



VII 



I 1.00 .28 .49 .51 J5 .43 39 

-21 .26 .31 .24 .37 

"{ S9 33 .48 37 

^ 1.00 .42 .52 .43 

J. 1.00 .46 .42 

^" 1 .00 



Note. — r > .20 p < .05. 



Multivariate Disciminaiit Analysis of the SCF 

A multivariate disciminani analysis (Vcldman, 
1967) was run on the STF data of ICK) ofikers. 90 
NCOs, and 99 airmen, by Fcderico U970) to 
determine how well the SCF distinguidied 
between the attitudes of the aforementioned Air 
Force rank groups. The data implied that the three 
groups have significantly different attitudes 
towards technical training, in that (a) NCOs have 
more favorable impressbns of technical training 
than do airmen, and (b) that officers have more 



positive attitude^ concerning training management 
and impressions than do airmen. Since the three 
groups differed in their altitudes towards technical 
training, meaningful interpretations of scores 
could be made only with refereiKe to the appro- 
prrate peer group, Le., normal respond patterns 
(norms) would be establidied ^arately for each 
group, thereby permHting SCF score inter- 
pretation within each of the three groups. Since 
the groups would be nomwd separately, the 
attitudes of officers would be compared with 
offKers in the norm group, NCOs with NCOs in 
the norm ^oup, and airmen with other airmen in 
the norm group 

Reliabaity 

Test-retcst reliability was .73 for the initial 
SCF, but later increased to .80 by adding fourteen 
items to the sc^e. The final, more reliable version 
of the SCF consisted, then, of 69 items repre- 
sentir^ seven factors. 

The Kuder-Richardson (1937) reliability, 
formula 21 (K-R 21) was computed on SCP scores 
of 1,685 students enrolled in technical training 
courses at (?hanute, Keesler. Sieppard, Lowry, 
Lackland, and Goodfellow AFBs. The internal 
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ci^nsistency reliabilities of the SCF for each of the 
six technical training schools; are presented in 
Table 3. Judging from these reliability coefficients 
it would appear that the SCF demonstrated 
ccnastent reliabflity at eadi of die six bases. The 
SCF responses of 1,669 students at die six h^ses 
listed above were also used to determine the 
rcliabaity of each of die seven scales of the SCF* 
The scale reliabilittes are presented in Table 4. An 
in^)ection of Table 4 indicates that most of the 
seven scales of the SCF are moderately reliable* 
the exception being Scale II (Study Environn^nt 
and Testing), The only rdiabflity coefficient above 
.SO in Scale 11 was at Lackland AFB at .S4. Several 
of the items in Scale II correlated h^ly with 
items it Scale III and therefore may explain the 
low internal con^ency of ihh scale. The nK>st 



reliable and con^stent ^ale was Sode I (Instructor 
Competence). 



TaNeS. K-R21 %FReUat^itiesforl»x Air 
Force Tedmicd Trakifaig Sclmc^ 





MnmlMf 




I Chanute 


214 


An 


II Keener 


333 


J&7S 


III Sheppard 


371 


J8S3 


IV Lowry 


523 


^83 


V Lackland 


182 


^1 


VI Goodfellow 


62 


.918 



Table 4. K-R 2 1 Reliab8ities of Seven Scales for Six Tedmkal Tmining Schools 



TBeiinleal Ttatnhifl Sdioots 





Chamttfl 


Kaetter 




Lowry 


LacMand 


O«0dfeNow 


1. Instructor Competence 


.73 


.78 


JO 


.82 


.78 


.83 


11. Study Environment and Testing . 


.37 


.49 


.47 


J8 


J4 


.47 


III. Specialty Training 


.63 


.73 


.65 


.67 


.70 


.73 


IV. Training Impressions 


.62 


.56 


J7 


.54 


.58 


.64 


V. Classroom Facilities and En%ironment 


.65 


.58 


J2 


M 


.72 


.72 


VI. Training Devices and Audio-Vist^l Aids 


.65 


.69 


,69 


.67 


.60 


.65 


Vii. Training Materials Adequacy 


j66 


.63 


.62 


.65 


.63 


.76 


Number of Subjects 


214 


333 


401 


523 


182 


62 



AdditioittI Facter Andysts of die SCF 

To determine the stabUity of the factor 
structure of the SCF, an additional factor anaty^ 
using a principal components ai^sis with a 
Varimax rotation (firman, 1967) was run on the 
SCF scores of !«669 students at the six technical 
training schools. The results of th» analysis 
reconfirmed die seven factors reported in Table 1. 
A few variations in factor loading were noted 
between the original and later factor analysis but 
were not considered to be of efficient magnitude 
to warrant a change in the original factor struc- 
ture. 

Validly 

Several estimates of validity have been 
determined by Federico (1970). The discrim- 
inative validity of die SCF was estaWislied in that 
• itlccrs. N('(5s. jfhl airmen were di-ilingui^able by 



their responses to ^me of the items in the ^F. 
That the SCF has sampling validity is apparent in 
that {a) the eritiqi^ form adquately sampled the 
objectiws of die stuctent critiqt^ prc^ram (ATCR 
52-29), and (b) Uie content for the items used in 
the S€F was generated by students enrolled in 
ATC technical training courses, not instructors or 
training managers. 

Skoiing IHt)cfidure 

The scoring system for the 7^2? is based on the 
assumption that students who make strong 
responses to questionnaire hems diould be differ- 
entiated from students who^ re^nses tend to be 
more moderate. The student re^KMids to die SCF 
by marking each statement in one of the categories 
of strongly agree (SA), agree (A), undecided <U), 
disagree (D), and strongly disagree (SD)- If the 
student marks *^SA'' on a positwe statement {the 
SCF is composed iif 52 positive and 17 negative 



statements), a score of 5 is given: '^A" is given a 
score of 4; and so on, to 3 score of . for "SD." 
The scoring system is reversed for negative items. 
A •*SA" response is given a score of I , and on, 
to a score of 5 for "SD*^ response. The Onal score 
for e^ student is obtained by aimming the 
individu^ item scores. The complete is 
presented in AppendU A along with the points to 
be assigned each response category within each 
item. 

A hi^ total or scale score ^nerally indk:atesa 
posith^e attitude towards technical training since 
such a score would result from agreeii^ with 
positive items and disagreeii^ with f^ti\^ items. 
The seven scales represent areas in the training 
environment that can be consktered r^tivdy 
independent from each other. As an example, a 
student might rate the instructors as competent 
but the instruction as repetitious. A situation such 
as this would tend to be diiguis^ by reporting 
SCF total score only, whereas, the subscore 
report would render this information readfly 
available. 



IIL PROCEDURE 

Adm&iktiatioii of ti^ SCF 

The SCF was administered between November 
1971 and January 1972 to 1,825 technical trainees 
taking courses at the following Tedinical Trataing 
Schools: Lowry, Lackland, Keesler, Sheppard, 
Oianute, and Goodfellow. The original sample was 
reduced to 1 ,669 due to errors hi re^nding and 
missing or inaccurate biograi^ic^ data. The ffaml 
sample consisted then of 994 airmen, 271 l«:Os, 
and 394 off^ers. Technk;^ training ccnirses were 
chosen on the t^as of their representativeness of 
the courses taught at each center; the distributfon 
of airmen, NCOs, and ofTfcers; and ud^ther 
students were at mid-course (within plus or minus 
one w^k of the middle of the course) or end- 
courM (within one week of die end of the course). 
Snce the minimum time any student in this 
»mple was enrolled in his re^thre tedink^ 
training course wn$ the mki-course point, it was 
assumed diat such a perkxl of time was long 
enough to develop attitu(tes toward Air Force 
technical training* 

Procedure 

The final version of the SCF, consKting of 69 
lAert-type items, was administered to die aib^cte 
in booklet form. The cover sheet of the booklet 
contained generalized information on the subject's 



task and a stater^nt insuring that attitudes would 
be held m strict confkience. The aibjects were 
instructed to read the statement on the cover Aeet 
ami wait for instructions before le^ndmg to the 
biographk:ai data. On page two the objects were 
requested to give bfographical informatk>n 
cof^ting of name, rairfc, socfail je<nirity number, 
AFSC, lei^h of servke, time in t^nk^ sdiod, 
technkal school enrollment, age, marital status, 
number of dependents, and levd of education. The 
objects also received instructioitt to report the 
AFSC they would be awards v^m conn>letk)n of 
the course and their course number. Sublets were 
instructed to re^t^ to the attitude statements 
beginning on page four. If an item dM not apjriy, 
subjects were ^ed to naik the **undeckicd*' 
categoiy. &ibject supervi^oi was piovkled by the 
test administrator and the dass instructor to 
prevent inter-subject collaboration. Testing time 
varied from twdve to thirty minutes induding the 
time ^nt reading instructions aisl any questions 
that were adced rdative to re^Kmding to the SCF. 
The SCF and its associated biogra{^ical data sheet 
is shown in Appendk A* 

TVeatiiientof ^SCF 

The cc^npleted surv^ were scored for obvious 
Inconastencbs in following directions and were 
coded in preparation for punching into IKM cards. 

As prevk)usly mentfoi^, the or^ral sample of 
1325 completed SCFs was reduced to 1,669 due 
to c^ain failures to foQow directions, and 
incorrect or inaccurate information. Re^c»ises to 
the sun^ were then purkjied into IBIA cards and 
verified. Data were cod^ for eadi of the items of 
information in th"^ biographical data sheet and also 
for ^ch stateme!it on the SCF* 

Scork^theSCF 

Scores were reported for eadi rank. Air Force 
Speciahy Code (AFSC), and Air Force Base where 
the technkal schod is located, as fatdlvidual 
avera^ scores arKl as scale ^ores. Hie indhidu^ 
average ^cne was derived by ccMnputbig total scoie 
on diie SCF usng conventicnml Likert scoring 
tedink}!^ and dhddinf tfiis figure by 69 (the 
number of ItenK in the SCF). Thfe score was 
rqx>rted oviy by category (rank, AFSC, base). 
&ale scores for ^ch of the se^n fetors that are 
listed in Table 1 were also reported for eadi of the 
thr J lank dasdficatbf^ Am!, ai^ base. Scsk 
scores were arrived at by totaling the responses for 
each scale and dividing by the number of items in 
the scale. Both total and ^ale scores were reported 
as a numi^r between ot^ and flw, whkh is the 
scoring range for each item in the SCF- 
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iV. RESULTS 

The average mean srores and accompanying 
standard deviaiions for 1,669 uftlcers, NCCh, at^ 
airmen are found in Table 5. An in^tion of 
Table 5 shows that all of the three groups pos- 
S'sscd attitudes that were above the **neutral*' 
point of 3 but less than the **agree** point of 4. 
The technical trainees in this sam{rfe mi^t then be 
described as ha^^ing attitudes towards technical 
training that were **sl^tly*' positive. A further 
Inaction of Table 5 revels that the most po^ti^^ 
attitudes were held by NCOs, followed by officers, 
and airmen. These results are in ger^ral agreement 
with the findings reported by Federico(1970). 

Table 5. Means ami Stai^ard Deviations of 
X¥ Scoies for 3 Hwk Gtoups 



Rank M^n so 



Officers 3.45 89 

NCOS 3.S3 .88 

Airmen 3.40 ,93 



Amb^ses of Varbnce of %F Scoies 

An analysis of variance was run to determine if 
the means for the three Air Force rank groups arc 
estimates of the same common peculation; Le. , arc 
the differences between the three m^ns the result 
of sampling error or are die means significantiy 
different and a re»dt of sampling funn three 
distinct populations? The resides of this analyds 
are fouml in Table 6. With the significance level set 
at .05, the critical value for significance with 2 and 
l,6S6 degrees of freedom was an F ratio greater 
than 19.50. Thus, it wss obvious that the F of 
47.03 was si^ificant at the .05 level of ccnfidence 
and that the differences between the means of the 
three rank groups were sufficiently great that they 
\^re assumed not to be estimates of a common 
population. However, it ^ould be noted that 
while this F is hi^ly significant, the actual group 
differences are small. The v^ue of co^ (Hiys, 
1963) is approximately .06, whidi indicates that 
only 6 percent of the mean square variance can be 
predicted by group member^ip. 



Source of Vlmtion 



Between Groups 
Within Groups 

Total 



Table 6. Anaiyss of Varbm^ for 3 Rank Giou{^ 



Sum of squares 



<lf 



Me«n Sqiiara 



137.19 
2417.76 

2554,95 



2 

1656 
1658 



68.57 
1.46 



47i)3* 



Note. — * p < .05. 



Kruskal^Wallb One-Way Analyst of Variance 
ofSCF Scores 

The knowledge that the three means in the rank 
category differed significantly, did not, however, 
reveal anything specific about the nature of the 
differences. What was required was a statistical test 
that would reveal how each mean differed from 
every other and whether there were significant 
differences between some of the means and not 
between others. The Kru^al-Wallis one-way 
analysis of variance by ranks was used to test the 
null hypothesis that the three rank group samples 
are a result of random sampling from the same or 
identical population with respect to avera^s. 
Since many of the assumptk)ns necessary for 
parametric measurement may not be met by the 



data in this study, a non-parametrk: test was used. 
The only asajmptions necessary for the Kruskal- 
Wailis test were that the variable under study had 
an underlying continuous distribution and the 
variables were at loast ordinally n^sured. !^nce 
the stron^y agree, agree« etc.« response categories 
were examples of ordinal m^surements with an 
underlying continuous distribution, it was a^umed 
that the data met the aforementioned minimd 
requirements. The results of the Kruskal-Wallis 
one-way analy^s of variarKe by ranks are 
pre^nted in Table 7. With the significance level set 
at .01 a \^lue of 9.21 or larger is r^uircd for 
significance at the .01 level and a value of 10^ or 
laiger is required for significance at the .005 level. 
An inspection of Table 7 reveals that all values are 
hi^ly significant at the .005 tevdjeading to the 
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conclusion that SCF scores of the three rank 
groups varied sigiifk-antly for each of the seven 
scales. The results were in agrccnwnt with those 
reported by Fcderito ( 1970) and as such pri>vided 
further evidence for norming the SCF separately 
for each of the three rank groups. 



Table 7. Noni^nmetric Andysesof Variattte 
of the Ak Force Kask Category 



SCF Scate Paetait ^ 


tuatttd 


1 


10.99 


II 


130.84 


III 


36.29 


IV 


37.13 


V 


38.02 


VI 


31.05 


VII 


39.22 



Note. — pH > 10.06 < ,005. 
*The vaUie derived from coni{Hicu^ the Kruskal- 
Wallis one way anaJysfe of variance by ranks. 



SCF Scafe Compar^ns by Rank C^^oty 

The scale me^ns, standard devatiwis, and 
response percentages for the 1,669 airmen, NCOs, 
and officers are found in Tables 8 throu^ 10. TTie 
means were the average means for each of the 
scales; e.g. , the scale I mean score was the m^n 
response for items 1 through 13, i;v4uch canprised 
scale 1, etc. The re^on» percentage was the 
percent of technical trainees who re^mted 1 
throu^ 5 for each scale. A cur»3ry review of 
Tables 8 throu^ 10 reveals that most re^n^ 
were in columns 3 and 4 with very few responses 
in columns 1, 2, and 5, indfcating fluctuations 
between the "undecided*" response and the 
**agree'* re^nsc. The most positive re^nses on 
Tables 8 through 10 were on Scale I (Instructor 
Competence) wherein 79 percent of officers, 
NCOs, and airmen manifested very po^tive 
responses towards the capability of the instructors. 
The **undecided" response wasu^d frequently by 
airmen. Over one half of die airmen sampled were 
undecided about Scale 2 (Study Environment and 
Testing), Scale 3 (Specialty Training), Scale 4 
(Training Impressions), and Scale 6 (Training 
Devices and Audio-Visual Aids). The only scale 
other tlian Instructor Competence that pioduced 
posithre results (over 1/2 of the sample •^agreed** 
or "stn)ngly agreed") was Scale 5 (Classroom 
Facilities and Environment). The overall indecision 



of the airmen in this sample may have i^ilected a 
lack of comparable cla^r(K}m experkmces with 
which to devdop adequate evaluation criteria. The 
NCC^ were only "undeciUcU" i>n Scales 2 and 3 
(Study Environment and Testing, and Specialty 
Training), respectively, which points to tlie 
po^bSity that their attitudes may have been xmte 
crystallized than the airmen in this sample. The 
NCOs showed more p«itive **agree*' resp<Mises 
than officers or airmen. O^r half of the NCOs 
marked "agree" on Scales 1 , 5, 6, and 7 excluding 
Scale 4, where 49 percent of the f^CH agreed with 
the po^tive staten^nts in that scale. It is inter- 
esting to note that none of the NCOs scored 1 or 2 
on the Instructor C(mipetence Scale, indicating 
generally favorable impresdons were formed by 
the NCOs of the competence of the instructor 
cadre. The re^onses of the officers are similar to 
some extent to the aim^n and NCOs, with the 
ex(^tk>n of their re^on^s on Scale 3 (Specialty 
Trasiliig). O^r 78 percent of the officers either 
"disagreed" i^ith the po^tWe items, **agrc^" with 
the negathre items or were "Hmd^ided" about the 
items in ^cialty Training. As were the airmm, 
over 50 perc^t of the ofllceR were •'undecided" 
on Scales 3, 4, and 6 (Sp^ialty Thiinkig, Training 
Impressions, and Repetitious Instructii^). Scales 
1, 2, 5, and 7 produced the most positive 
re spoils for officers with over SO percent 
delaying favorable attitudes towards items in 
those scales. 

Tables. Scale Mearo, SD, and Re^nmse (%) 
for Airm» (yV=994) 



Item S€pr« 





1 


2 


3 


4 


9 


Maan 


SD 


1 


0 


I 


17 


75 


7 


3M 


.44 


2 


0 


5 


65 


30 


0 


3.21 


.44 


3 


0 


11 


SS 


30 


1 


3.12 


.57 


4 


0 


8 


S6 


36 


0 


3.28 


.51 


5 


0 


2 


42 


56 


I 


3.48 


•44 


6 


0 


3 


62 


36 


0 


3.29 


.40 


7 


0 


5 


47 


48 


1 


3.41 


.47 



The only scale that pn^uc^ unfavorable 
responds with all of the three rank groups was 
Specialty Training. Many of the items In Scale 3 
concern the adequacy of the course as a prep- 
aratk)n for actual on the job performance. Since 
many of the students were unfamfliar with the 
actiol field job requirements, they were unable to 
make an informed estimate of the relevance of the 
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Table 9, Sc . Meai^S0*s,8iid Re8poi»e(%) 
for NCOS (Ar»271) 







AM»ORte (%) 








Item Score 




















2 




4 


s 


Mean 


so 


1 


0 


0 


13 


79 


8 


3.93 


42 


2 


0 


I 


63 


37 


0 


3J3 


38 


3 


0 


9 


S8 


31 


1 


3.18 


.55 


4 


0 


6 


44 


49 


1 


3.43 


.54 


5 


0 


2 


28 


66 


3 


3.63 


.45 


6 


0 


3 


42 


53 


2 


3.44 


.46 


7 


0 




35 


61 


2 


3.56 


.47 


Table 10. 


Scak 




SD% and ResNmse (%) 




for Officers (A'^ 394) 


















item Seere 
















Scale 


1 


2 


3 


4 


s 




so 


1 


0 


1 


IS 


78 


4 


3.83 


.43 




0 


1 


44 


55 


0 


3.49 


.36 


3 


2 


21 


55 


22 


0 


2.92 


.62 


4 


1 


7 


62 


30 


0 


3.19 


SI 


5 


0 


1 


20 


69 


1 


3J9 


.37 


6 


0 


3 


59 


38 


0 


3.31 


.41 


7 


0 


2 


35 


63 


1 


3.54 


.42 



that many of the students were neutrd towards 
technical training as mea^i^ by ihe and 
these data must be ccMisidercd the vaseline from 
whkh norm comparbons wflt nMide. 

Tabkll. AttktHi^ ExfM'f'ssed by the Hiree 
Ak Fwce Gfoui» on the %F 



Rank 














Officers 


51 


43 


6 


iCOs 


57 


4C 


3 


Aimwn 


45 


SO 


5 



^Avttst^ mcati score of 3. 
^Average mean score of 1 cm* 2, 

SCF Score Cmnp^!&aas by Gtf^cHy 

The awiage mean scores and accompanying 
standard deviations for the SCF scores of the six 
technical training schools (Lackland, Sieppard, 
Keener, Clianute, Lowry ami GoodfeQow AFBs) 
are ^own in Table 1 2. As was the case with the 
thr^ Air Force rank categories, all six base groups 
poss^d attitudes, as mea^red by the SCF, 
towards technkral trafaiing that could be descried 
as "di^tly poritive/* The most podthre attitudes 
were expressed by students at Base while tl^ 
most negative attitudes were found at Base F. 



trainfaig they h^ receh^d. However, all of the 
three groups manifest^ positive attitudes to^rds 
Scales 1 and 5 (Instructor Competence and 
Qassroom Faclittes and Environn^t) with Scale 
7 (Training Materials Adequacy) running a close 
third. The percentage of ofi^r^ HCOs and 
airmen thai manifested positive (an aver^ m^n 
score of 3), and negatK^ (an average mean score of 
1 or 2) atUtucfes on the SCF is fhown in Table 1 1. 
As was pfevk>udy noted, the held the most 
positive attitudes towards technic^ tramii^ 
followed 1^ the officers and the NC(H« It is 
unclear why the '^undect<ted'* category was used 
frequently. Perhaps these responses w^e due to a 
de^re on the part of some students to c<Hnplete 
the SCF rapidly without r^dii^ die iten^ 
carefully. Students who felt di^tly posith^ or 
sli^tly neptive towards an ^teTT did not judge 
that their filing was strong rnou^ to mark the 
agree or disagree re^>onse. Pcrh;ips more explicit 
instructicHis on the meaning of eadi response 
category may have evoked more agree and di^gree 
responses on the SCF. The fact remains, thou^, 



Table 12. SCF Aver^ I^an Scorns for Six 
Tedioicai Trainfaig Schoob 



ftgaait SD 

A 3.43 .90 

B 3.46 ,90 

C 3.49 .90 

D 3J7 .93 

E 3.52 ^9 

F 3J0 .96 



Analy^ of V^^mnce of SCF Scores 

An analysis of varbnce was run to test for 
signifkant difference between the bases. The 
results of th^ analysis can be fotmd in Table 13. 
With the signifii^ni^ level set at .05, an F ratk> 
greater than 4J6 is si^fkant with 5 and 1,709 
degrees of freedom. &nce the obtan^d F mto of 
9X)1 was si^iffcant at tl^ JOS levd of confidence, 
it was conduded that the differences between the 
means of the six base groups were not lai^ 
cnou^ to be accounted for solely on the basis of 
chance. 
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Tahlv II 



Ansiym of Variance of Six Technical Tniinfaig Schools 



Source of VariAtion 


Sum of Squares 






F 


Between Groups 
Within G^utps 




5 

1709 
1714 


35.97 
3.97 


9.01* 


P < .01 . 



Kni^.Walib Oiw Way-Analysis of Vartance 
ofSCF Scoies 

As with the Air Force ranks cafegt>ry, the 
Kruskal-Wallis tost was run to determine the 
extent to which each mean differed from every 
other and whether or not there were signiHcant 
dilTerenccs between some means and not between 
othere. The null hypothesis stated that the six base 
group samples were a result of random samplir^ 
from the same or identical populations with 
respect tu averages. The results of the Kruskal- 
Wallts one-way analysis of avariance by ranks arc 
presented in Table 14. A value of 16.7 or larger 
was required for significance at the .005 level of 
c"onfidence. Inaction of Table 14 reveals that ail 
of the values were highly significant at the .005 
level. The results suggest that the SCF scores for 
the six ba«s varied significantly on each of the 
seven scales of the SCF. Thus, it is evkient that 
due to the significant differences between the 
scores at the six technical schools, the SCF should 
be normed separately for each base, /.e, compar- 
isons between SCF scores would use data compiled 
for students at that base only. The differenc-es in 
attitudes towards technical training between the 
six bases may have been due to several diverse 
factors. Students may have had higher morale at 
stmte bases that provided superior facilites and 
more opportunities for off-base entertainment. 
Another important influence may have been the 
Air Force Specialty Cjxlft^AFSC). Since s^wie 
AFSCs are obviously more^pular" than others. 



students who were enrolled in relatively 
"unpopular" AFSCs may have reflected n»ore 
dissatisfied responses on the SCF than similar 
students enrolled in a more pi)pular course. 



Table 14. 



Non-parametric Analyses of Variance of 
the Air Force CategMy 



SCF %cm Pacterf 



H «ttu«' 



i 

11 
111 

IV 
V 
Vi 

VII 



42.55 
28.24 
33.55 
63,45 
40,90 
53,91 
61,47 



Notc-pH> 16.7 <, 005. 

♦ 

SCF Scate Compiff^ns by Base Categoiy 

The scale means and standard -jeviations for the 
six technical training schools arc found in Table 
15. The hi^u^ scale mean for each base was 
consistently Instructor Competence. The only 
mean above 4.00 was found at Base E on (Scale 1 ). 
The most negative re^nse was also consistent 
across each base for Specialty Training, The large 
number of undecided and native responses on 
Scale 3 aiems to indicate again, that students were 
basically unsure, especially arimen, as to whether 
their training was actually preparing them to 



perform adequately in the ffeld 
Tabids. SCF Scale Means ( MN) for Six Technic^ Training Schools 



Bate 



B 



Scales 



MN SO MN SO Mtg so MN SO MN SO 



MN SO 



Instructor Competence 3 85 

Study Fjivironment and Testing 3.33 

Specialty Training 3 05 

Training Impre^uns 3 29 

CIassr<M>m Facilities and Environment ^.49 

Training Devices and Audio-Viaial Aids 3.35 

Training Materials Adequacy 3.37 



,39 
.40 
i7 
.54 
.44 
.40 
.48 



3.82 
3J7 
3.1 1 
3.29 
364 
3.29 
3.54 



.44 

.43 
.61 
.51 
J9 
41 
44 



3.90 

3.32 

3.11 

333 

3,51 

341 

3.57 



,40 
41 
.57 
.51 
.38 
42 
.42 



3,81 
3.24 
3S)l 
3.18 
3.48 
3.26 
3.3S 



45 
43 
.55 
.50 
45 
43 
47 



4.01 
3J6 
3.26 
349 
3.52 
341 
3.52 



42 
47 
.60 
49 
47 
J5 
42 



3.70 
3J1 
2.92 
3.11 
3.40 
3.24 
329 



S2 
.42 
.69 
.59 
.50 
41 
.57 
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V. CONCLtSIONS AND RECOMMENDATIONS 

The n?sults of the factor atialysis. using the data 
from Phase II of the effort iv develop a new 
student critique form for the Air Tiainii^ 
Command, basically confirmed the factor struc- 
ture reported by Federico (1970). However, Scale 
8 (Textboi^k Utility), as reported by Federico 
(1^70), was eliminated. The items left over from 
the foregoing scale were placed into other scales 
based upon factor headings and subjective inter- 
pretations of degree of fit to other ^ales. The 
scales were then renamed, to aid interpretation of 
their content by instructors and training managers. 

Based upon the results of the KR-2I reliability 
measure of the internal consistency of the 
subscorcs i)f the SCF, several items are being 
studied for modifKration or elimination. An item 
will be eliminated if its inclusion in a subscore 
lowers the internal consistency of the ^bscore, if 
the content of the item re^ts in a dispropor- 
tionate number of undecided re^nses, or if the 
content of the itStn^ppears to be inappropriate. 
Thus, several revisions and item eliminations will 
be studied prior to recommending the SCF for 
wide^read ATC usage. 

The analysis established that officers, NCOs, 
and airmen displayed significanily different 
attitudes towards technical training on all ^vcn 
scales of the SCF, althou^ the differences 
between ilie SCF scores of the three groups were 
relatively small. Significant differences were also 
found between the sbc bases on each scale of the 
SCF. While these significant differences were the 
justification for norming the SCF separately by 
rank and base, it must be remembered that with a 
sampie as large as the oi^ reported in this study 
that only small differences in SCF ^^ores were 
required for significance. Thus, even thou^ 
signifKrwt differefwes were found, the differences 
between the SCF scores of the base and rank 
categorfcs were not large. However, the data did 
seem to indicate that NCOs had the most favorabte 
attitudes towards technical training, followed by 
officers, and airmen. 

The next phase in the itevdopment of an 
improved critkjue program entaOs the devel- 
opment of the appropriate computer software to 
manage the entire critique progra:n within Air 
Ftuce technical training by administering scoring, 
and interpreting the SCF. The comjHiter softw^'C 
presently being developjd wfll provide a score, 
rcfjpnnsc percentage, and norm comparison for the 
liital scale and each subscale of ? SCF. A report 



format is being devel(^>ed that will prwide 
training ccmtmanders. supervisors, and instructors 
with all of the pertinent infom^tion they will 
need to {a) determine how attitudes towards 
technic^ training of the stu(knts in their class 
compare with the attitudes of ^udents of a simOar 
rank at the same technical training center, (b) 
identify **felt'* problem areas, ami (c) ictentify 
areas in technk^ traming that ate receivkig a 
favorable re^nse. As a result, stu(tent feedback 
can be obtained in a nH>re efficient, automated 
manner, thereby relie^g instructors ar^ trailing 
managers of the drudgery of interpretmg and 
summarizing the aucfent critk^iie forms as they 
presently exist. 



VL^MMARY 

1 . The uses and limitations of student critkjues 
as a tool for the valuatkin of technic^ training 
courses were discussed. It was suggested that 
studer.; critiques can be quite reltable and effective 
for course evaluation. Several stud^s that 
demonstrated the utflity of using students to rate 
the effectiveness of instructors and course content 
were also discussed* 

2. Relevant critieria for selecting or con- 
striK:tii% an approprfaite ^udent critkiue 
instrument were listed* A brief histoiy of the 
devdopment of the prototype critkiue form was 
presented along with evidenwe that demonstrated 
the relfability and validity of the prototype form. 
Additiorml items were added to improve the 
reliability of the sc^e wtth the result that the 
present Stuttent Critique Form (SCF) coi^ists of 
69 items contained in ^ven sc^es. 

3. As the results of a previous study had 
su^ested, the SCF was norn^d separately for 
ofikers« NCOs, and airmen to acccmimodate for 
significant differences on the part of these three 
groups with resp^t to ^vei^l distinct factors of 
training. Data from the administratton of die SCF 
to 1,669 students enrcdled in technit^ training 
courses at Lowry, Ladcland, Keeskr, Chanute, 
I9ie{^drd,and Goodfellow AFBs was used to norm 
the SCF sei^rately by rank and base. 

4. Subsequent analyses indicated again that the 
rank and base groups were significantly different 
and ^ould« therefore, be normed separately as was 
su^t^ by Federko (1970). An additk)nal 
factor atmly^s bask:ally reconfirm^ the seven 
scales that were developed previously. The 
impiementatk>n of Phase III of die prt^ct to 
dc\^*lop a new sfudeni critkiue form was also 
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discussed. The implementation of the program to 
develop the computer software to manage the 
entire student critique program was feciimnwnded 



and has since been initiated. Upon compleiion of 
Phase in of this project, the SCP will be ofTercU t«> 
the Air Training Command for operational use. 



RtFKRfcNtliS 



ATC Regulation 52-29. I^udeni critique program. 
Randolph Air Force Bass, Tex.: Department of 
the Air Force, 19 July 1970. 

Anflterf, AM. Factors affecting student evaluation 
of college faculty membert. Jmntal of Appffed 
PsychoU/g}: 1953. 48S-460. 

Goffman W.E. Determinii^ students' concepts of 
effective teaching from their ratings of in- 
structors. Journal of KducathruU Psychology, 
1954,45,277-286. 

Cosgrove, DJ. EWagnostic rating of teacher 
performance. Jtxtmd of Educathtwl Psy- 
chology. 1959.50,200-204. 

Edwards, A. Techniques of attitude con- 
stmctUm. New York: Apirfeton-Century-Crofts 
Inc., 1957. 

Federko, P.A. DevH<^ment of psychometric 
measures of student attitudes towml technicd 
training: Relubdity md factork^ v^iditv 
AFHRL-TR-70-37, AD723 314. Lowiy AFB, 
Coio.: T^hnicd Trainii^ Divisfon, Air Force 
Human Resources Laboratory, November 1970. 

Federico. P.A. Identifyir^ item vdiditv indices 
utilizing a multivaiate model. AFHRL-TR- 
71-16, AD.729 763. Lowry AFB, Colo.: 
Technical Training Division, Air Force Human 
Resources UtK)ratoiy, April 197 1 (a) 

Fecferico. P.A. Degree of evaluative a^rtbns 
ascribed to an attitude universe as a function of 
measurement format, f^ychologkd Reports, 
1971,29, 1315-1324Xb) 

Hannan, H.H. Mtidern fxtor amlym. (2nd ed.) 
Chicago: The University of Chkago Press. 
1967. 



Hays, W.L. Statistics for psychologists. New York: 
Holt, Rinehart, and Winston. 1963. 

Hdlamler, E.P. A validity of peer nomination in 
predicting a dbtant performance criterion. 
Jmmd of Applied Psychol(^v. 1965 , 49, 
434-438. 

KudCT, G.F. & Richaidson. M.W., The theoiy of 
the estimation of test rdiabflity. Psvcfi- 
ome/TOS. 1937,2, 151-160. 

L&ert, RA. A technique for the meaairenient of 
attitudes. Archives of hychology, 1932, 22, 
(140), 1-55. 

McKrachK, W J. & Solomon, D. Student ratings of 
instructors. Journal of Edmationd Rexunh. 
1958,51,379-382. 

^>ecial Evaluation Braiwh, Training Evaluati<-n 
Divfcion, 3415th Technical School, fwiftwrttw 
of criteria used by students in completing A TC 
Form 7J6. "Student Critique," Lowiy AFB. 
Colorado, 1967. 

Spencer, R.E., & Al^moitf, L3A., The illimm 
course evduation questionnaffe: A dexription 
of its development and a report of some irs 
remits. Research Re^Kxct No. 292. Measurement 
and Research ttvsion, Office of InstrucUonal 
Rcrources, University of lllinofe, 1971. 

Veldman, D. Fortran prngmmming for tht- 
behavhral sciences. New York: Holt, Rinehart. 
and Winston, 1%7. 

Weawr, CJl. Instructor rating by college students, 
Jourrud of Fducationcd PsyrholoKy. I960, 51, 
21-25, 



!6 



AmSDIX A. ATC FORM 736, STUDENT CRITIC'JE 



STUDENT CRITIQUE 



COURSE NO. 



COURSE TITLE 



GRADE 



DATE 



CLASS MO. 



FROM 



PEWIQD or TR AiNiNG 



SHIf T 



TO 



SOUADRON 



NSTRwCTiQNS: Se^ow ^te a se'tes of items wt^icn can t>e rated OytstamSmg ^0)« Sattsfdctoty cS) 0^ Unsatisfactory (U^ CHecK i ^ \ the 
eacfi ite-^ wntch best expresses wur oo«n«on. If you late an item unsatisfactory <U), give specl^c conwents and fecamme«dMion$ 



ITEMS 


COMMENTS 


RATINC 


0 


S 


u 


1. INSTPUCTIQN ass ContfoL Attitude, Entnustasn*, 

Hei3fu^"e5<i. Ur^ile^standaDie^ 










-ND'V'OUAL ASSISTANCE: > Rerr^edjal (f^tt ruction , 
Coii'^se 










TFjAfN^NG VETHOOS* Af^ount of Tneoiy & Practical, 
Us«» c^f Tfjjntng Ti'^e. Student f'drttCioatJOn^ 










4. TRAlNiNG LtTERATURE; • Avail. t&u tv. Use and 
*^eic**.''^e$&. Stucs^'^t Study Guides, i^o^kt>ooks. Technics 

Orde«*. V.in<j.ils. T»»xtDO0i*S 










f . VISUAL AfOS' 'AvaUa&ility, Usv and Helrfu!ness. 
^ TcansTJtencies, C^a^ts^ 










6. T^AiNtNG EQtMWENT: . Avai'.3&«i*ty, USe & He^oful- 
x*ss. Sf ste'^s Eaj p-^ent. Test Eau p-ent, Tools^ 




1 
1 






\ W<?<TTEN AND PRACTICAL TESTS ^Under Stma^le, 
A-?^ rifStfit on. Cr t OLie- 


1 

1 


1 




..„■■ ,1 


CLASSROOM AMD TRAINING AREA: -L Heat, 
A'- V fc^ir-Ant, Mc Lewel' 


i 
1 








1 

V OVERALL CVAU'JATION OP COURSE | 

\ 











ffOBM PPEVO IS COITIOK MAY BE USEE UNTIL STOCK IS EKHAUSTEO. 

MAv -rj ,7 ou.S.aovefnrt.ont Printing 0«f ice: 1972 779.}06/a8 
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APPt:NDIXB: PROPOSED STUDENT CRJIIQUE FORM 



Air Force Ibaan Resources Laboratory 
Technical Training Division 
Lowry Air Force Base 



This borxlet contains stateaents designed to measure student 
attitudes toward Mr Force technical training. It Is Intended to 
give students In Air Force technical schools an opportunity to 
express their real feellons about what can be done to laprove this 
training. Please read the following stateaents and respond truth- 
fully to each. The Infosaatlon obtained will be used solely for 
research. Y .ur attitudes will be maintained in strict confidence. 
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Naae _______________ SSAN _______ 

Present Grade APSC 

Total length of service In tlie Air Force: 

yearn «ontas 
tour Technical School 

Nus&er of aonths in Technical School _ 
Niad>er of aonths In preaoit grade 

Your age in fears on your loBt birthday 

Harltal status: Single ( } Married ( ) 

Dl«*orced or Vldonr^d ( ) 
Nunber of dependents 

Educatltm: Circle one nunber lowing highest level reached. 

1. SosM grasJBar school (did not finish) 

2. GrasBBar school graduate . 

3. Sone high school (did not finish) 

4. High school graduate 

5. Sone college (less than tuo years) 

6. Two or arre years of college, but no degree 

7. College 'iegree 

8. Graduate work, but no degree 
9« Masters degree 

10. Doctors dsigrea 



19 



PUaae re^d each of the following ieeaa and Indicate the 
aaount of your agreenent or ulsagreeaane with its contents. 
Point out the ^..r^f ^. ^.. ^ you are of the saate op inion Ky 
making a check uark under the appropriate itea. 

EXAMPLE: Most of the tiaa Beetle Bailey enjoys basic 
military trailing. 

Strongly Dls- Un- Agree strongly 

Di«- agree decided ^».«^on«Ay 



agree 

< > ( ) ( ) ( ) 



Agree 
( ) 



« yo" strongly disagree with this stateaent, you should 
place a check «ark (✓) under strongly dls« ^r ee to show th.r 
you are of a very different op in<nn. if ^gree with this 
statement . you should place a check aark (✓) under agree to 
show that you are of the same op lnto n > If you have no op ini»n 
about the statement or if you are und«^<ri^ about its contents, 
you should place a check aark (✓) under undecided . 
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1. Most of your instructors aeea to kaov thalr subject ratter. 

Strongly Olf- Un- AgrM Stroi^ly 

Dls- agree decided Agree 
agree 

( ) ( ) ( ) ( ) ( ) 



2. Most of your technical instruccors seesi to be well educated. 

Strongly Dls- Un- Agree Strongly 

Dis- agree decided iVgree 
agree 

( ) ( ) ( ) ( > ( ) 



3. Most of your technical instructor's classroCT presfmtati<ms are 
well organised. 

Strtmgly Agi^.e Ifa- Dis- Strcmgly 

Agree decided agree Ois- 

( ) ( ) ( ) ( ) < ) 



^. Most of tYk& tiae your teclmicai instructor's dassrooe preaoita- 
tions are hard to understand. 

Strongly Dis- Un- Strongly 

Dis- agree decided Agr^ 

agree 

( ) ( ) ( ) ( ) ( ) 



5 Most of your uechni^al instructor's presentations are clarified 
by exoeples ar.d illustrations. 



Strongly Agree tte- Dis- Strongly 

Agree decided agree Dis- 

^ree 

( ) ( ) ( ) < ) < ) 



ERIC 
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Host o£ v3ur technical insttoctors stem interested In their 
subject sLitter. 



Strongly Agree Un- Dls- Strongly 

Agree decided sgree Dis- 

agree 

( ) ( ) ( ) ( ) < ) 



Host of the tine your instructor thoroughly explains new technical 
nateria'' . 

Strongly Agree Un- Dis- Strongly 

Agree decide agr^ Dis- 

agree 

( ) ( ) ( ) ( ) ( ) 



Host of the training aaterials seeas telated to course objectives, 

Strongly Agree Un- Dis- Strongly 

Agree decided agree Dis- 

agree 

( ) ( ) ( ) < ) < ) 



Most of the tiae you are informed of the training objectives of 
each diss session. 

Strongly Dis- Un- Agree Strongly 

Dls- agree decided Agree 

agree 

( ) ( ) ( ) ( ) ( ) 



Host of your technical sctool Instructors seen to be experienced 
teachers. 

Stronglv Dis- Un- Agree Strongly 

Pis- agree decided Agr^ 

agree 

( ) ( ) ( ) ( ) C ) 
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n. 



Hose of the tl&e your inacructor glvas yoix individual help with 
difficult technical B«teriAl. 



Strongly Agree Un- Die- Strongly 

Agree decided agree Dis- 

agree 

( ) i ) < ) ( ) < ) 



12* Most of your technical Instructors encourage class participation. 

Strongly Dis- Un- Agree Strongly 

Dis- agree decided Agree 
agree 

( ) ( ) ( ) ( ) ( ) 



13 « Most of the time your techalcal instructor evades answering 
questions asked during clasa. 

Strongly Agree Un-- Dis*- Strongly 

Agree decided agree Dis* 

agree 

( ) ( ) ( ) ( ) C ) 



14« Most of the time you have enough tisie during technical school 
class days f .r individual study. 

Strongly Agree Un- Dis- Strongly 

Agree decided agree Dis* 

agree 

( ) ( ) ( ) ( ) ( ) 



15 • Most of the time your barracks are quiet enough for studying* 

Strongly Agree Un- Dis- Strongly 

Agree decided agree Dis- 

agree 

( ) ( ) ( ) ( ) ( ) 



^ 23 

ERLC 



Most of the tiae additional dxu ies you are assigned Interfere 
with your study. 

Strongly Dls- Un- 

Dls- agree dec Med 
agree 

< ) C ) ( ) 



Most of the time you have to wait one or more weeks before you 
know what your score la on a particular technical school test. 

Strongly Agree Un- Dls- Strongly 

decided agree Dis- 

agree 

< > C ) ( ) ( ) ( ) 



Agree Strongly 
Agree 



( ) ( ) 



Most of the time you are not given enough tlae to finish your 
technical school tests. 

Strongly Dls- Un- Agree Strongly 

Dls- agree decided Agree 

agree 

^ > c ) ( ) ( ) ( ) 



Host of your scores on written tests reflect how well you will 
be able to perform your USAF Job specialty. 

Strongly DLs- Ua- Agree Strongly 

Dls- agree decided Agree 

agree 

( > < > ( ) ( ) ( ) 



Moat of the time Instn ctlonal television Is used in your 
technlcrl ^.£a^n^llg. 

Strongly Agree Un- Dls- Strongly 

Agree decided agree Dis- 

agree 

< > C ) ( ) ( ) ( ) 
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21. Most of the tlrc you need individual aasistmice to learn 
teclmical material. 



Strongly Agree Un- Dls- Strongly 

Agree decided agree Dis- 

agree 

( ) : ) ( ) < ) ( ) 



22 1 Most of the time in technical school you are pressed to learn 
material at a faster rate than you are capable. 

Strongly Dis- Un- Agree Strongly 

Dis- ag*-ee dec Ided Agree 
agree 

( ) ( ) ( ) ( ) ( ) 



23. Most of the t.^me you should be given additional tests within a 
block in technical school. 

Strongly Dis- Uh- Agree Strongly 

Dis- agree decided Agr^ 

agree 

( ) i ) ( ) ( ) ( ) 



24. Most of the time in technical school six hours of class each 
working day ±3 tiresome. 

Strongly Ois- Un- Agree Strongly 

Ois- agree decided Agree 

agree 

< ) ( ) ( ) ( ) ( ) 



25. Host of your formal education was considered by the Air Force in 
assigning you to a particular technical school. 

Strongly Agree Un- Dis- Strongly 

Agree decided agree Dis- 

agree 

( ) ( ) ( ) ( ) ( ) 
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K )8t of your tralnliis oacatlala actually teach you hov to perfora 
y mr nev UHAF job specialty* 

S^.rong'.y Dls- Un- Agree Strongly 

CIS- agree decided Agree 

a^ree 

< ) < ) ( ) ( ) ( ) 



V'jBt o*' what you are taught in techcical school will help you 
get a hettnr civilian Job. 

Szrongly Dta- Un- Agree Strongly 

Ei^- a«jrea decided Agr^ 

a jree 

( > < ) < ) ( ) ( ) 



Kqs<: technical school classroom lectures help you dt^velop the 
rjw VSi\F skill you are trying to learn. 

S'^-ongXy Dws- Un- Agree Strongly 

I^iS' a:;ree decided Agree 

a'^rise 

( ) < ) ( ) ( ) ( ) 



Mi5sc o\ yoltr teclinical Instructors motivate you to laam your 
LSAF speci/ilty. 

Strongly Agree Un- Dis- Strongly 

decided agree Dis- 

agree 

f ) ( ) ( ) ( ) ( ) 



Most ni your okills are being properly used by the Air Force. 

Stronf.ly Dis- Un- Agree Strongly 

Dis- a^t&Q decided Agree 
agree 

< ) ( ) ( ) ( ) ( ) 
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Most of ywa scohm od p«rfonuBiee taats rftfleet Iwv veil you 
will be Able Co perfoxs your D$A7 job specialty. 

Strongly Agree rsn- Die- Strongly 

Agree decided egree Die- 



< > ( ) ( ) ( ) ( ) 

Most of tbe tlae in t ech n i cal school the training is so reglaoited 
%.hat the leaming eniriroiSBent is diainished. 

:itrongly Oia- Jki- ilgree Strongly 

I^is' a&ree decided Agree 
agree 

' > ( ) C ) ( ) ( ) 

Most of the f.lme your student squadron sets aside each class night 
enough tiae for sleep. 

Strongly Agree tta- Dis- Strongly 

Agree decided agree Dis- 



( ) ( ) ( ) ( ) ( ) 



Most of the time topics vlthln your technical course follow a 
logical 8eqi«eiice* 

Strongly Agree Un- Die- Strongly 

Agree decided agree Dis- 

agree 

( ) ( ) ( ) ( ) ( ) 



The length of your technical course is just right to pr^are 
you for your USAP job specialty. 

Strongly Agree Un- Dis- Strongly 

Agree decided agree Dis- 

agree 

< ) ( > ( ) ( ) ( ) 
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36 



Host of the tUB& the pace of Inatructlon within your technical 
course Is fast ^o«tgh to i&eep you froa being bored. 



Strongly Die- On- Agree Strongly 

Ois- Agree decided Agree 
agree 

( ) ( ) ( ) ( ) ( ) 



37. Most of tae tlae your technical instructors use nev and interesting 
training methods to teach you your course materials. 

Strongly Dis- Uh- Agree Strongly 

Dis- agree decided Agree 
^igree 

( ) ( ) ( ) ( ) ( ) 

38. >tost of the tism class critique of your technical school tests 
helps ya« learn the course materials better. 

Strongly Dis- Oh- Agree Strongly 

Dis- agree decided Agree 
agree 

( ) ( ) ( ) ( ) ( ) 

39 • Most of the time technical school classrooms are too small for 
the nus&er of students in a class. 

Strongly Agree Un- Dis- Strongly 

Agree decided agree Dis* 

agree 

( > ( ) ( ) ( ) ( ) 

AO. Most of the tlae classroom temperatures are adequately maintained. 

Strongly Dis- Un- Agree Strongly 

Ois- agree decided Agree 
agree 

( ) ( ) ( ) ( ) ( ) 
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41. Mqsc e£ thA tim eX«a 



lifhta arc brl^lit •aeitgh* 



Strongly 
Agree 



( ) 



AgrcA 



( ) 



Un- 
decided 



( ) 



Die- 
agree 

( ) 



Strongly 
Die- 
agree 

( ) 



42. Moet of your claeeroMM are properly ventilated. 

Die- 



Strongly 
Die-' 
agree 

( ) 



C ) 



declded 



( ) 



( ) 



Stro&gly 



( ) 



43. Most of the tlae the aeating arrangeront vithin your claaaro<»B 
^llowa you to adequately eee the inatroctor and the chalkboard. 



Strongly 
Agree 



( ) 



i^ee 



< ) 



decided 



( ) 



Di8- 



( ) 



Strongly 
Die- 
agree 

( ) 



44. Moat of the tiira you have eaay accasa to copiee of AF r^ulationa, 
si^uale, and technical ordera require aa aupplesoitary reading 
in your courae. 



Strongly 
Agree 



( ) 



Agree 



( ) 



Un- 
decided 



( ) 



Die- 
agree 

( ) 



Strcmgly 

Dia- 

agr^ 

( ) 



43. Most of the tixae you have eaay acceaa to hands-on traiAere for 
extra practice dt*riag off-duty hours. 



Strongly 
Dis- 
agree 



Dis- 
agree 



Un- 
decided 



Agree 



Strongly 
Agr^ 



( ) 



( ) 



( ) 



( ) 



( ) 
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Host of 9he timm ehmSxm aad tables used la your dasarooas are 
coafortabla eaotagh* 

Strongly Die- Un- Agree Strongly 

Dia- agree decided Agree 

agree 

C > ( ) ( ) ( > ( ) 

Host of the tlae there are too i&any atudenta using one trainer. 

Strongly Agree Un- Dla- Strongly 

decided agree Ois- 

agree 

^ > ( > ( ) ( ) ( ) 

Most of the tlse the solae in your technical achool classroom 
Is aalntained at a aialBun. 

Strongly Agree to- dis- Strongly 

AS^^ decided agree Dis- 

agree 

< > ( ) ( ) ( ) ( ) 

Most training films help you to better understand the technical 
subject natter. 

Strongly Dis- On- Agree Strongly 

Dis- agree decidtsd Aaree 

agree ^ 

< ) < ) ( ) ( ) ( ) 

Most training films and slide presentations motivate you to learn 
technicrl material. 

Stronglr- Dis- On- Agree Strongly 

Wa-^ agree decided Agree* 

< > ( ) < ) ( ) ( ) 
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Most trftlBl&g imtiiam thmt you iim bAlp to b«tt«r understasd 
Qflitf concepts. 

Strongly Dla- tfa- AgtM Strongly 

Dls- Agreo doeiM Agree 

agree 

< ) c ) C ) ( ) ( ) 



Hoet paper-and -peaoll teste thet you ere given is. tecimlcel 
school thoroughly cover the toplce. 

Strongly Agree tta- Dis- Stroi^ly 

Agree decided agree Dis- 



< > ( > ( ) ( ) < ) 



itost course days there ie enoi^h tlae aUoned for you to practice 
on/with hands-on training aids. 

Strongly Dis- Qs- AgrM Strongly 

Die- agree decided Agree 

agree 

( ) ( ) ( ) ( ) ( ) 



Most of the different kinds of training eids/e^ipismt used 
In technical school are evaileble for your self-^tikly. 

Strongly Dis- tUs- Agree Strongly 

Dis- agree decided Agree 
agree 

< ) < > ( > ( ) ( ) 



Most training filas used in your technical «>ttrse help you learn 
laportant facts about performing your USAF job specialty. 

Strongly Agree tti- »is- Strongly 

Agree decided agree Dis- 

agree 

( > < ) C ) ( ) ( ) 
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Most of t*iA tim eloMd-eircttlt TV It n Mmcttw ttAiolag 
ctehiiltiii*. 

screagly WlM" m- A^zmm Strongly 

Dis- agr«a d«eld«d AgrM 

itgree 

< ) < ) ( ) ( ) ( ) 

AccordlAg to your iaatruetor Boot o£ tha ttAiciag f Urns you see 
are out-of-date by tha tlM you aaa thea. 

Strongly Agree Ife- Die- Strongly 

Agrae decided agree Dis- 

agTM 

^ ) ( > < > ( > ( > 

Heat of the niaa thara are too aaay atndaata praetlelng on one 
training device. 

Stnmgly Die- Ua- Agree Strongly 

I)l8- agree decided Agree 

ffgree 

( ) ( ) ( > ( ) ( ) 

Most of your technical instruction is spent vietdLng training filas. 

Strongly XWs- th»- Agree Strongly 

ttiB" agree decidmi Agr^ 
agree 

( > ( ) ( ) ( > ( ) 

Moat of the tiae your technical training equi^^aHAt is not 
operation tl« 

Strongly Die- tti- Agree Strongly 

Dis- agree decided Agrea 

agree 

« > ( ) ( ) ( ) ( ) 
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61* Tott e«a tadwrstMtf sMt of fcmt craiaiBS Mt«rial«. 

Strongly Agre* te- OU- StrengXy 

Agree decided egree Die- 

agree 

< ) ( ) ( > ( ) ( ) 



62. Hoet of the vrlttea teete yon receive la teelmlcal school are 
eaay to tmderstaad. 

Strongly Dla- Ub- Agree Stroi^ly 

)Hs- agree decided Agree 

agree 

( ) < ) ( ) ( ) ( ) 



63. Hoet of your studeat etudy guides are eaay to uaderstaad. 

Strongly Agree Ha- X>l8- Stroagly 

AgrM decided agrM Dls- 

i^rM 

( ) ( ) ( ) ( ) ( ) 



64. Moat of the tl»e your instructor saist suppleaeat the training 
materials because he says they are not current* ^ 

Strongly Dls- t^- Agree Strongly 

Dls- agree decided Agree 
agree 

( ) ( > ( ) ( > < ) 



65. Host of the time your Instructor refers you to saterlal which 

suppleaeats your tralalag guide. 

Strongly Ois- fo- Agree Strongly 

Dls- agree decided Agree 

agree 

( ) ( ) < ) ( > ( ) 
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Komt of the InfonattMi la your trolatag aaterlala Is uo-eo-date 
•ad aeenrMe. •>-»•» 

Strongly DU- Un- i 

MJ^ Mr- d«.M^ 



< > < ) < ) { ) 



( ) 



Mojt of ywr ttalalag Mtwtals hava «ao«uh lUiMtntioiio to 
l»Xp yea lean the taptee. viiiwsratioM to 

S»e»,ly MM- it,. Agre. gtw^i, 

< > < ) ( ) < ) < > 

SS^gS'd^h'JS SSJ!*"*^ ^ 

strongly igrea On- Die- strongly 

deeidnl agraa Dt»- 

^ > < > c ) < > ( > 

to Ad«4iMt«lsr eovar th« eourM. 

Strongly Pio- ut|. Agro. 8^^^,,^^ 

■gVM 

< > < > < > < ) < ) 
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APPENDIX C: STUDENT CRITIQUE F(HIM ITEM RE^NSES 



Table CI. hem M«aas, SDs, and Respoase ^nventi^ for Akmen (A^s994) 
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Table CI (Continued) 
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Table CJ. ttem Ikteim. SDs, asd Re^oase Fnc^ta^ fw (Mfkers (A^>394) 
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